Human eye controlled computer mouse interface

ABSTRACT

A system, method, and device are provided that receive an image sequence from a camera, processes the image sequence, extract gaze information, map the user&#39;s eye movements to movement of the mouse cursor, and initiates mouse clicks based on user behavior.

TECHNICAL FIELD OF THE INVENTION

The technology of the present disclosure relates generally to a system,method, and device for a human eye controlled computer mouse interface,and more particularly to a system and method for controlling a mousecursor and mouse clicks utilizing face and eye detection withoutphysical movement of the mouse.

BACKGROUND

A computer mouse is typically used as a pointing device to move a cursoron a computer display to select, move, or manipulate objects using mouseclicks. The computer mouse has been through a variety of technologicaladvancements, such as physical changes, modifications and enhancementsin the features, etc. However, it has remained difficult for physicallydisabled people to use a computer mouse. Recent advancements haveattempted to solve this problem with limited success due to high cost,low efficiency, unreliability, health risks, inability to handle smallmovements of the pupil or changes in the target's position, the need forspecial hardware, and other issues.

This disclosure provides a means to aid individuals with physicaldisabilities and all common users in using a computer. A system isprovided that enables controlling a mouse cursor using the human eyewithout the use of additional sophisticated hardware.

SUMMARY

A system, method, and device are provided that receive an image sequencefrom a camera, processes the image sequence, extract gaze information,map the user's eye movements to movement of the mouse cursor, andinitiates mouse clicks based on user behavior.

According to one embodiment of the present invention, there is provideda computer device comprising an eye-controlled computer mouse interfacesystem, the computer device comprising:

a camera or multiple cameras; a display; a memory;

a processor programmed to execute a program comprising theeye-controlled computer mouse interface system, wherein theeye-controlled computer mouse interface system comprises a dataacquisition function, a calibration function, an eye detection function,a face detection function, and a gaze mapping function;

wherein the data acquisition function is configured to:

-   -   acquire data in the form of at least one of individual images,        stored video, live streaming video, or live streaming video from        multiple cameras, wherein the acquired data is stored as a        sequence of images, and individual images in the sequence of        images are referred to as frames; and    -   set a current frame in the image sequence, wherein a first frame        in the image sequence is set as the current image;

wherein the calibration function is configured to:

-   -   determine ratio values used to map movement of a user's eyes to        the corresponding movement of a mouse cursor on the display; and        store the ratio values in the memory;

wherein the face detection function is configured to:

-   -   detect a face in each frame of the sequence of images; extract        the detected face; store the extracted frame in the memory; and        end processing of the current frame if the user's face is not        detected;

wherein the eye detection function is configured to:

-   -   retrieve the extracted face; localize and extract an eye region        encompassing both of the user's eyes from the extracted face;        split the eye region into a right eye region and a left eye        region; determine for each eye region an eye center and a pupil        center; store the eye center and pupil center for each eye        region in the memory; and    -   detect a left eye wink, a right eye wink, or a blink, wherein        the left eye wink is a closing of the left eye while the right        eye remains open, the right eye wink is a closing of the right        eye while the left eye remains open, and the blink is a closing        of both the left eye and right eye;

wherein the gaze mapping function is configured to:

-   -   map movement of the user's eyes to movement of the mouse cursor        on the display using the eye center and pupil center for each        eye region retrieved from the memory and the ratio values        retrieved from the memory;    -   perform a left mouse click when the left eye wink has been        detected for a number of frames greater than or equal to a wink        threshold; perform a right mouse click when the right eye wink        has been detected for a number of frames greater than or equal        to the wink threshold; and perform a double left mouse click        when the blink has been detected for a number of frames greater        than or equal to a blink threshold.

According to one embodiment, the calibration function is furtherconfigured to: display a target in a range of selected positions on thedisplay, wherein the user follows the target on the display; initiatethe eye detection function to determine the pupil center and eye centerfor each target position; determine the ratio values using the pupilcenter, eye center, and target position for each of the targetpositions; and store the ratio values for the user in the memory.

According to one embodiment, the gaze mapping function furtherconfigured to: utilize a four or eight direction algorithm to mapmovement of the user's eyes to movement of the mouse cursor on thedisplay.

According to one embodiment, the calibration function is furtherconfigured to: determine if the user is a new user or if the ratiovalues were previously determined for the user and stored in the memory;and retrieve the ratio values if the ratio values were previouslydetermined for the user and stored in the memory or determine the ratiovalues for the new user.

According to one embodiment, the eye detection function furthercomprising tracking eye regions in the current frame when the eyeregions were detected in the previous frame, wherein tracking comprisespredicting the location of the eye in the current frame using the eyecenter in the previous frame.

According to one embodiment, the eye detection function uses a pointdistribution model of the face.

According to one embodiment, the eye detection function detects a blinkor wink by fitting a straight line to corners of the eye and top andbottom of the eyelids as determined by the point distribution model ofthe face, wherein the wink or blink is detected if the line has an errorless than an error threshold.

According to one embodiment, the gaze mapping function is furtherconfigured to map movement of the mouse cursor based on the location ofone eye and correct the mapping using the corresponding localization ofthe second eye.

According to one embodiment, the right eye region and left eye regionare used to perform a distance measurement of a user from the displayand to compensate for movement of the user's head.

According to one embodiment, the eye detection function localizes theeye by fitting an elliptical curve to the eye region.

According to one embodiment, the system is further configured tocontinuously track the gaze of the user using live stream data.

According to one embodiment, a sleep instruction is sent to the deviceif a blink is detected for a number of frames greater than or equal to asleep threshold.

According to one embodiment, the resolution of the device camera is 1.3megapixels or greater.

According to another embodiment of the present invention, there isprovided a method for controlling an eye-controlled computer mouseinterface, the method comprising:

-   -   acquiring data in the form of at least one of individual images,        stored video, live streaming video, or live streaming video from        multiple cameras, wherein the acquired data is stored as a        sequence of images, and individual images in the sequence of        images are referred to as frames;    -   setting a current frame in the image sequence, wherein a first        frame in the image sequence is set as the current image;        determining ratio values used to map movement of a user's eyes        to the corresponding movement of a mouse cursor on a display;    -   storing the ratio values; detecting a face in each frame of the        sequence of images; extracting the detected face; storing the        extracted frame; ending processing of the current frame if the        user's face is not detected; retrieving the extracted face;    -   localizing and extracting an eye region encompassing both of the        user's eyes from the extracted face; splitting the eye region        into a right eye region and a left eye region; determining for        each eye region an eye center and a pupil center;    -   storing the eye center and pupil center for each eye region;    -   detecting a left eye wink, a right eye wink, or a blink, wherein        the left eye wink is a closing of the left eye while the right        eye remains open, the right eye wink is a closing of the right        eye while the left eye remains open, and the blink is a closing        of both the left eye and right eye;    -   mapping movement of the user's eyes to movement of the mouse        cursor on the display using the retrieved eye center and pupil        center for each eye region and the retrieved ratio values;    -   performing a left mouse click when the left eye wink has been        detected for a number of frames greater than or equal to a wink        threshold; performing a right mouse click when the right eye        wink has been detected for a number of frames greater than or        equal to the wink threshold; and performing a double left mouse        click when the blink has been detected for a number of frames        greater than or equal to a blink threshold.

According to one embodiment, the method further comprises: displaying atarget in a range of selected positions on the display, wherein the userfollows the target on the display; determining the pupil center and eyecenter for each target position; determining the ratio values using thepupil center, eye center, and target position for each of the targetpositions; and storing the ratio values.

According to one embodiment, the method further comprises: utilizing afour or eight direction algorithm to map movement of the user's eyes tomovement of the mouse cursor on the display.

According to one embodiment, the method further comprises: determiningif the user is a new user or if the ratio values were previouslydetermined for the user and stored; and retrieving the ratio values ifthe ratio values were previously determined for the user and stored ordetermine the ratio values for the new user.

According to one embodiment, the method further comprises: tracking eyeregions in the current frame when the eye regions were detected in theprevious frame, wherein tracking comprises predicting the location ofthe eye in the current frame using the eye center in the previous frame.

According to one embodiment, the eye region localization is performedusing a point distribution model of the face.

According to one embodiment, the method detects a blink or wink byfitting a straight line to corners of the eye and top and bottom of theeyelids as determined by the point distribution model of the face,wherein the wink or blink is detected if the line has an error less thanan error threshold.

According to one embodiment, the method is further configured to mapmovement of the mouse cursor based on the location of one eye andcorrect the mapping using the corresponding localization of the secondeye.

According to one embodiment, the right eye region and left eye regionare used to perform a distance measurement of a user from the displayand to compensate for movement of the user's head.

According to one embodiment, the method localizes the eye by fitting anelliptical curve to the eye region.

According to one embodiment, the method is further configured tocontinuously track the gaze of the user using live stream data.

According to one embodiment, the method further comprises sending asleep instruction if a blink is detected for a number of frames greaterthan or equal to a sleep threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

Features that are described and/or illustrated with respect to oneembodiment may be used in the same way or in a similar way in one ormore other embodiments and/or in combination with or instead of thefeatures of the other embodiments.

FIG. 1 is a schematic view of an exemplary system for controlling mousecursor movement and clicks utilizing face and eye detection withoutphysical movement of the mouse; and

FIGS. 2A-2F are flow diagrams representing exemplary actions taken byvarious components of the system of FIG. 1.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments will now be described with reference to the drawings,wherein like reference numerals are used to refer to like elementsthroughout. It will be understood that the figures are not necessarilyto scale.

With reference to FIG. 1, illustrated is a schematic block diagram of acomputer device 10. The computer device 10 may comprise a desktopcomputer, a laptop computer, a tablet computer, a smart phone, a digitalvideo recorder (DVR), a television, a DVD player, or other similardevice. The computer device 10 includes an eye-controlled computerinterface system 12 that is implemented using computer technology. Theeye-controlled computer interface system 12 is configured to execute adata acquisition function 40, a calibration function 42, a facedetection function 44, an eye detection function 46, a gaze mappingfunction 48, and to store a database 16.

In one embodiment, the data acquisition function 40, calibrationfunction 42, face detection function 44, eye detection function 46, andgaze mapping function 48 are embodied as one or more computer programs(e.g., one or more software applications including compilations ofexecutable code). The computer program(s) and/or database 16 may bestored on a machine readable medium, such as a magnetic, optical orelectronic storage device (e.g., hard disk, optical disk, flash memory,etc.).

To execute the data acquisition function 40, calibration function 42,face detection function 44, eye detection function 46, and gaze mappingfunction 48, the eye-controlled computer interface system 12 includesone or more processors 18 used to execute instructions that carryout aspecified logic routine(s). In addition, the eye-controlled computerinterface system 12 has a memory 20 for storing data, logic routineinstructions, files, operating system instructions, and the like. Asillustrated, the calibration function 42, data acquisition function 40,face detection function 44, eye detection function 46, gaze mappingfunction 48, and database 16 may be stored by the memory 20. The memory20 may comprise several devices, including volatile and non-volatilememory components. Accordingly, the memory 20 may include, for example,random access memory (RAM), read-only memory (ROM), flash devices and/orother memory components. The processor 18 and the components of thememory 20 may be coupled using a local interface 22. The local interface22 may be, for example, a data bus with accompanying control bus orother subsystem.

The eye-controlled computer interface system 12 may have various videoand input/output (I/O) interfaces 24. The video and I/O interfaces 24may be used to couple the device to a display 28, camera 30, mouse 32,and/or keyboard 34. The display 28 may be a computer monitor, televisionscreen, mobile phone display, or other suitable display. The camera 30may be a webcam, video camera, digital camera, network camera, highdefinition (HD) camera, security camera, wireless camera, cameraintegrated into the computer device 10, series of multiple cameras, orother suitable camera. The camera 30 may have a resolution of at least1.3 megapixels.

The eye-controlled computer interface system 12 may have one or morecommunications interfaces 26. The communications interfaces 26 mayinclude, for example, a Bluetooth module, a Zigbee module, a wirelessUSB module, a USB connector, a mini-USB connector, a firewire IEEE 1394connector, micro-USB connector, and/or other standard ornon-standard/proprietary protocols.

With reference to FIGS. 2A-2F, illustrated are logical operations toimplement exemplary methods of controlling mouse cursor movement andmouse clicks utilizing face and eye detection without physical movementof the mouse. Executing an embodiment of the eye-controlled computerinterface system 12, for example, may include carrying out the followingexemplary methods. Thus, the flow diagram may be thought of as depictingsteps of one or more methods carried out by the eye-controlled computerinterface system 12. Although the flow charts show specific orders ofexecuting functional logic blocks, the order of executing the blocks maybe changed relative to the order shown, as will be understood by theskilled person. Also, two or more blocks shown in succession may beexecuted concurrently or with partial concurrence.

With continued reference to FIG. 2A, the eye-controlled computerinterface system 12 calls the data acquisition function 40, calibrationfunction 42, and face detection function 44. The eye-controlled computerinterface system 12 calls the eye detection function 46 and gaze mappingfunction 48 if a face is detected by the face detection function 44 instep 162. If a face is not detected by the face detection function thenthe eye-controlled computer interface system 12 ends processing of thecurrent frame and begin processing the next frame in the image sequence.

With reference to FIGS. 2A and 2B, the data acquisition function 40receives a sequence of images 150, wherein the individual images in thesequence of images may be designated as frames. The data acquisitionfunction 40 sets the first frame in the image sequence as the currentframe in step 152. The sequence of images may be in the form ofindividual images, stored video data, live streaming video data, livestreaming video data from a series of cameras, or data in other suitableform. The image sequence may comprise images captured by the camera 30or images stored in the database 16. The data acquisition function 40receives live streaming video data from the camera 30 when theeye-controlled computer interface system 12 is used to control a mousecursor in real time.

With reference to FIGS. 2A and 2C, the calibration function 42determines the parameters necessary to map movement of a user's eyes tothe corresponding movement of the cursor on the display 28. Theparameters may comprise ratio values r_(x) and r_(y). The calibrationfunction 42 performs calibration for each user, because mapping themovement of a user's eyes to movement of a cursor on the display 28 isuser dependent. In step 130, the calibration function 42 determines ifthe user of the computer device 10 is a new user. The calibrationfunction 42 may determine if a user is a new user by performing facerecognition, prompting a user for identification, by assuming thecurrent user is the same user who last used the computer device 10, orany other suitable method.

If the current user is determined to be a previous user, the ratiovalues for the user are retrieved from the database in step 136. If thecurrent user is determined to be a new user in step 130, a target isdisplayed across a range of selected positions on the display for theuser to follow in step 131. The ratios r_(x) and r_(y) are calculatedfrom the data collected as the user follows the movement of the targeton the display 28 in step 132.

For example, the ratios used to map the movement of the eye to the sizeof the display may be estimated using the following equations:

$r_{x} = \frac{x_{ec} - x_{e}}{x_{sc} - x_{s}}$$r_{y} = \frac{y_{ec} - y_{e}}{y_{sc} - y_{s}}$where x_(s) and y_(s) are, respectively, the x-axis and y-axis locationof the cursor on the display; x_(sc) and y_(sc) are, respectively, thecenter of the display 28 in the x-direction and y-direction; x_(ec) andy_(ec) are, respectively, the eye center in the x-direction andy-direction; and x_(e) and y_(e) are, respectively, the location of thepupil center in the x-direction and y-direction.

The position of the target (x_(s) and y_(s)) during calibration and thecenter of the display (x_(sc) and y_(sc)) are known. The eye center(x_(ec) and y_(ec)) and pupil center (x_(e) and y_(e)) are measured bythe eye-controlled computer interface system 12 as described below.Using these values, the ratios r_(x) and r_(y) mapping movement of theeye(s) to the size of the display in the x-direction and y-directionrespectively can be determined.

After determining the ratio values in step 132, the ratio values for theuser may be stored in the database or memory in step 134.

With reference to FIGS. 2A and 2D, the face detection function 44 isinitiated following step 42. The face detection function 44 performsface detection on the current frame in step 160. The face detectionfunction 44 may use the Viola-Jones method, Neural Network-based method,statistical based method, shape based method, example based learningmethod, wavelet based method, Probabilistic Modeling of Local Appearancemethods, Spatial Relationships based methods, or any other suitablemethod for face detection. If a face is detected, the face detectionfunction 44 extracts the user's face from the current frame and storesthe extracted face in the memory 20 or database 16.

If a face is not detected in the current frame in step 162, processingof the current frame ends and the current frame is incremented to thenext frame in the image sequence in step 112. Conversely, if a face isdetected in the current frame in step 162, the detected face region isextracted in step 164 by the face detection function 44 and the eyedetection function 46 is initialized.

With reference to FIGS. 2A and 2E, the eye detection function 46 detectswinks and blinking to initiate mouse clicks. A wink is a closing ofeither the left or right eye. A blink is a closing of both eyes.

In step 180, the eye detection function 46 sets the current eye as theleft eye, retrieves the extracted face from the memory or database, andresets the left eye wink Boolean, right eye wink Boolean, and blinkBoolean to false. Next, the eye detection function 46 may check if thecurrent eye was detected in the previous frame of the image sequence instep 182. If the current eye was detected in the previous frame, eyetracking may be performed on the extracted face region from the facedetection function 44 in step 184. Eye tracking tracks movement of thecurrent eye between frames by predicting movement of the eye to aspecific region. Eye tracking in step 186 may be performed using a colorbased particle filter or other suitable means. Eye tracking may begin atan initial point, wherein the initial point is the eye center localizedin the previous step.

If the current eye was not detected in the previous frame of the imagesequence in step 182, the eye detection function 46 localizes the eyeregion in step 186. The eye region includes both the left and righteyes. The eye detection function 46 may use a model based eye detectionframework, Hierarchical wavelet network based methods, Neural-networkbased methods, statistical based method, shape based method, SpatialRelationships based methods, or other suitable method to localize theeye region. The right and left eyes may be localized by fitting anelliptical curve to the eye region. The estimates of the fit ellipseroutine can be correlated with the shape distribution of the eye ifknown.

Following localization of the eye region in step 186, the eye detectionfunction 46 splits the eye region to extract both the right eye regionand left eye region in step 188. If the current eye is detected in theprevious steps in step 200, the blink counter and the wink counter forthe current eye is reset to zero in step 202. The blink counter andcurrent eye wink counter are reset to zero, because the eye was detectedand not closed during a wink or blink. Following step 202, the currenteye center is calculated in step 204. Next, in step 206 the pupil centeris calculated. The pupil may be detected using model based detection,color based detection, or other suitable detection methods. The pupilcenter and eye center for the user may be stored in the database ormemory.

If the current eye is not detected in step 200, the current eye winkBoolean is set to true in step 209. The eye detection function 46 mayalso perform blink/wink detection using a point distribution model ofthe eye region. The eye detection function 46 may utilize a shape modelor other suitable means to detect the corners of the current eye andtop/bottom of the eye lids. The eye detection function 46 may attempt tofit a straight line to these points in step 207. The current eye blinkBooleans may be set to true in step 209 if the eye detection function 46finds a straight line fit with an error less than an error threshold instep 208.

In step 212, the current eye is checked to determine if the current eyeis set to be the right eye. Both eyes have been analyzed if the currenteye is set to be the right eye, because the left eye is analyzed beforethe right eye. Therefore, the right eye has not been analyzed if thecurrent eye is set to be the left eye. In step 213, the current eye isset to be the right eye and the previous steps are repeated if thecurrent eye is set to be the left eye.

If the current eye is set to be the right eye in step 212, the value ofthe left eye wink Boolean and right eye wink Boolean are checked. If theleft and right eye wink Boolean values are both set to true in step 214,then a blink has been detected and the blink counter is incremented byone in step 216 and the eye-controlled computer interface system 12exits the eye detection function 46.

If either of the left eye or right eye wink Boolean is set to false 214,the value of the left eye wink Booleans is checked 220. If the left eyewink Boolean is set to true, the left eye wink counter 222 isincremented by one and the eye-controlled computer interface system 12exits the eye detection function 46.

If the left eye wink Booleans is set to false in step 220, the value ofthe right eye wink Booleans is checked in step 224. If the right eyewink Boolean is set to true in step 224, the right eye blink counter isincremented by one in step 226 and the eye-controlled computer interfacesystem 12 exits the eye detection function 46. If the right eye wink

Boolean is set to false, the eye-controlled computer interface system 12exits the eye detection function 46.

Following the eye detection function 46, the gaze mapping function 48 isinitialized. The gaze mapping function 48 maps movement of the user'seyes to movement of the cursor on the display in step 240. A four oreight direction algorithm may be used to map movement of the user's eyesto movement of the mouse cursor on the display. For example, movement ofthe mouse cursor can be correlated to movement of the user's eyes basedon the following equations:x _(s) =x _(sc)((x _(ec) −x _(e))*r _(x))y _(s) =y _(sc)((y _(ec) −y _(e))*r _(y))

The ratios (r_(x) and r_(y)) are estimated as described above andretrieved from the memory or the database, the display center (x_(sc)and y_(sc)) is known, and the eye center (x_(ec) and y_(ec)) andlocation of the pupil center (x_(e) and y_(e)) are measured by theeye-controlled computer interface system 12 as described above andretrieved from the memory or the database. The location of the mousecursor on the display 28 is updated using this information and the aboveequations.

In the above equations, x_(ec), y_(ec), X_(e), and y_(e) corresponds todata calculated for the right eye only or may correspond to datacalculated for the left eye only. For example, if a wink is detected,x_(ec), y_(ec), x_(e), and y_(e) calculated from the open eye may beused. If both eyes are open, an average value of (x_(ec)−x_(e)) and(y_(ec)−y_(e)) for the left eye and right eye may be calculated and usedto determine xs and ys in the above equations. A large difference in(X_(ec)−X_(e)) and (y_(ec)−y_(e)) for the right eye and left eye maysignify an error in eye detection, resulting in the gaze mappingfunction 48 discarding the data determined for the current frame in theprevious steps. The mouse cursor may remain unchanged from the previousframe if a blink is detected.

Additionally, if both eyes are detected, the distance between the lefteye and right eye may be used to calculate a change in distance of theuser from the display. The change in distance of the user from thedisplay may be used to modify the ratio values in order to account forthe change in distance of the user from the display.

The gaze mapping function 48 also performs mouse clicks based on theresults of the blink/wink detection performed by the eye detectionfunction 46. The detection of a right eye wink, left eye wink, or blinkcorresponds to different mouse clicks. For example, a left eye winkinitiates a left mouse click, a right eye wink initiates a right mouseclick, and a blink initiates a double left mouse click by the gazemapping function 48. Conversely, the mouse click assigned to aparticular eye wink or blink may also be set by the user.

Additionally, a sleep instruction can be initiated to put the computerdevice 10 into sleep mode if an extended blink is detected. An extendedblink may comprise a blink that lasts, for example, greater than 120frames, or other suitable number of frames.

A right eye wink, left eye wink, or blink is determined based on theleft eye wink Boolean, right eye wink Boolean, and blink Boolean. Theblink counter corresponds to the number of consecutive frames that ablink (closing of both eyes) is detected. The left eye and right eyewink Boolean corresponds to the number of consecutive frames that theleft eye and right eye are closed respectively.

A double left mouse click is initiated in step 244 if the blink counterreturned by the eye detection function 46 is greater than a blinkthreshold in step 242. The blink threshold may be, for example, 5frames, 10 frames, 15 frames, or any suitable number of frames. Inanother embodiment, the blink threshold may be set by the user.

If the blink counter is not greater than or equal to the blink thresholdin step 242, the left eye wink counter is compared to the wink threshold246. If the left eye wink counter is greater than or equal to the winkthreshold 246, a left mouse click 248 is initiated. The wink thresholdmay be, for example, 5 frames, 10 frames, 15 frames, or any suitablenumber of frames. In another embodiment, the wink threshold may be setby the user.

If the left eye wink counter is not greater than the wink threshold instep 246, the right eye wink counter is compared to the wink thresholdin step 250. If the right eye wink counter is greater than or equal tothe wink threshold in step 250, a right mouse click is initiated in step252.

The eye detection function 46 is closed if the right eye wink counter isless than the wink threshold in step 250, or following the initiation ofa double left mouse click in step 244, left mouse click in step 248, orright mouse click in step 252.

Following execution of the gaze mapping function 48, the current frameis incremented to the next frame in the image sequence in step 112 andprocessing is repeated.

Although certain embodiments have been shown and described, it isunderstood that equivalents and modifications falling within the scopeof the appended claims will occur to others who are skilled in the artupon the reading and understanding of this specification.

What is claimed is:
 1. A computer device comprising an eye-controlledcomputer mouse interface system, the computer device comprising: acamera or multiple cameras; a display; a memory; a processor programmedto execute a program comprising the eye-controlled computer mouseinterface system, wherein the eye-controlled computer mouse interfacesystem comprises a data acquisition function, a calibration function, aneye detection function, a face detection function, and a gaze mappingfunction; wherein the data acquisition function is configured to:acquire data in the form of at least one of individual images, storedvideo, live streaming video, or live streaming video from multiplecameras, wherein the acquired data is stored as a sequence of images,and individual images in the sequence of images are referred to asframes; and set a current frame in the image sequence, wherein a firstframe in the image sequence is set as the current image; wherein thecalibration function is configured to: display a target in a range ofselected positions on the display, wherein the user follows the targeton the display; initiate the eye detection function to determine a pupilcenter and eye center for each target position; determine ratio valuesusing the pupil center, eye center, and target position for each of thetarget positions, wherein the ratio values are used to map movement of auser's eyes to the corresponding movement of a mouse cursor on thedisplay; and store the ratio values in the memory; wherein the facedetection function is configured to: detect a face in each frame of thesequence of images; extract the detected face; store the extracted framein the memory; and end processing of the current frame if the user'sface is not detected; wherein the eye detection function is configuredto: retrieve the extracted face; localize and extract an eye regionencompassing both of the user's eyes from the extracted face; split theeye region into a right eye region and a left eye region; determine foreach eye region the eye center and the pupil center; store the eyecenter and pupil center for each eye region in the memory; and detect aleft eye wink, a right eye wink, or a blink, wherein the left eye winkis a closing of the left eye while the right eye remains open, the righteye wink is a closing of the right eye while the left eye remains open,and the blink is a closing of both the left eye and right eye; whereinthe gaze mapping function is configured to: map movement of the user'seyes to movement of the mouse cursor on the display using the eye centerand pupil center for each eye region retrieved from the memory and theratio values retrieved from the memory; perform a left mouse click whenthe left eye wink has been detected for a number of frames greater thanor equal to a wink threshold; perform a right mouse click when the righteye wink has been detected for a number of frames greater than or equalto the wink threshold; and perform a double left mouse click when theblink has been detected for a number of frames greater than or equal toa blink threshold.
 2. The system of claim 1, the gaze mapping functionfurther configured to: utilize a four or eight direction algorithm tomap movement of the user's eyes to movement of the mouse cursor on thedisplay.
 3. The system of claim 1, wherein the calibration function isfurther configured to: determine if the user is a new user or if theratio values were previously determined for the user and stored in thememory; and retrieve the ratio values if the ratio values werepreviously determined for the user and stored in the memory or determinethe ratio values for the new user.
 4. The system of claim 1, the eyedetection function further comprising tracking eye regions in thecurrent frame when the eye regions were detected in the previous frame,wherein tracking comprises predicting the location of the eye in thecurrent frame using the eye center in the previous frame.
 5. The systemof claim 1, wherein the eye detection function uses a point distributionmodel of the face.
 6. The system of claim 5, wherein the eye detectionfunction detects a blink or wink by fitting a straight line to cornersof the eye and top and bottom of the eyelids as determined by the pointdistribution model of the face, wherein the wink or blink is detected ifthe line has an error less than an error threshold.
 7. The system ofclaim 1, wherein the gaze mapping function is further configured to mapmovement of the mouse cursor based on the location of one eye andcorrect the mapping using the corresponding localization of the secondeye.
 8. The system of claim 1, wherein the right eye region and left eyeregion are used to perform a distance measurement of a user from thedisplay and to compensate for movement of the user's head.
 9. The systemof claim 1, wherein the eye detection function localizes the eye byfitting an elliptical curve to the eye region.
 10. The system of claim1, wherein the system is further configured to continuously track thegaze of the user using live stream data.
 11. The system of claim 1,wherein a sleep instruction is sent to the device for the device toenter a sleep mode if a blink is detected for a number of frames greaterthan or equal to a sleep threshold.
 12. The system of claim 1, whereinthe resolution of the device camera is 1.3 megapixels or greater.
 13. Amethod for controlling an eye-controlled computer mouse interface, themethod comprising: acquiring data in the form of at least one ofindividual images, stored video, live streaming video, or live streamingvideo from multiple cameras, wherein the acquired data is stored as asequence of images, and individual images in the sequence of images arereferred to as frames; setting a current frame in the image sequence,wherein a first frame in the image sequence is set as the current image;displaying a target in a range of selected positions on a display,wherein the user follows the target on the display; determining a pupilcenter and eye center for each target position; determining ratio valuesusing the pupil center, eye center, and target position for each of thetarget positions, wherein the ratio values are used to map movement of auser's eyes to the corresponding movement of a mouse cursor on adisplay; storing the ratio values; detecting a face in each frame of thesequence of images; extracting the detected face; storing the extractedframe; ending processing of the current frame if the user's face is notdetected; retrieving the extracted face; localizing and extracting aneye region encompassing both of the user's eyes from the extracted face;splitting the eye region into a right eye region and a left eye region;determining for each eye region the eye center and the pupil center;storing the eye center and pupil center for each eye region; detecting aleft eye wink, a right eye wink, or a blink, wherein the left eye winkis a closing of the left eye while the right eye remains open, the righteye wink is a closing of the right eye while the left eye remains open,and the blink is a closing of both the left eye and right eye; mappingmovement of the user's eyes to movement of the mouse cursor on thedisplay using the retrieved eye center and pupil center for each eyeregion and the retrieved ratio values; performing a left mouse clickwhen the left eye wink has been detected for a number of frames greaterthan or equal to a wink threshold; performing a right mouse click whenthe right eye wink has been detected for a number of frames greater thanor equal to the wink threshold; and performing a double left mouse clickwhen the blink has been detected for a number of frames greater than orequal to a blink threshold.
 14. The method of claim 13, wherein themethod further comprises: utilizing a four or eight direction algorithmto map movement of the user's eyes to movement of the mouse cursor onthe display.
 15. The method of claim 13, wherein the method furthercomprises: determining if the user is a new user or if the ratio valueswere previously determined for the user and stored; and retrieving theratio values if the ratio values were previously determined for the userand stored or determine the ratio values for the new user.
 16. Themethod of claim 13, wherein the method further comprises: tracking eyeregions in the current frame when the eye regions were detected in theprevious frame, wherein tracking comprises predicting the location ofthe eye in the current frame using the eye center in the previous frame.17. The method of claim 13, wherein eye region localization is performedusing a point distribution model of the face.
 18. The method of claim17, wherein the method detects a blink or wink by fitting a straightline to corners of the eye and top and bottom of the eyelids asdetermined by the point distribution model of the face, wherein the winkor blink is detected if the line has an error less than an errorthreshold.
 19. The method of claim 13, wherein the method is furtherconfigured to map movement of the mouse cursor based on the location ofone eye and correct the mapping using the corresponding localization ofthe second eye.
 20. The method of claim 13, wherein the right eye regionand left eye region are used to perform a distance measurement of a userfrom the display and to compensate for movement of the user's head. 21.The method of claim 13, wherein the method localizes the eye by fittingan elliptical curve to the eye region.
 22. The method of claim 13,wherein the method is further configured to continuously track the gazeof the user using live stream data.
 23. The method of claim 13, whereinthe method further comprises sending a sleep instruction for a device toenter a sleep mode if a blink is detected for a number of frames greaterthan or equal to a sleep threshold.
 24. A computer device comprising aneye-controlled computer mouse interface system, the computer devicecomprising: a camera or multiple cameras; a display; a memory; aprocessor programmed to execute a program comprising the eye-controlledcomputer mouse interface system, wherein the eye-controlled computermouse interface system comprises a data acquisition function, acalibration function, an eye detection function, a face detectionfunction, and a gaze mapping function; wherein the data acquisitionfunction is configured to: acquire data in the form of at least one ofindividual images, stored video, live streaming video, or live streamingvideo from multiple cameras, wherein the acquired data is stored as asequence of images, and individual images in the sequence of images arereferred to as frames; and set a current frame in the image sequence,wherein a first frame in the image sequence is set as the current image;wherein the calibration function is configured to: determine ratiovalues used to map movement of a user's eyes to the correspondingmovement of a mouse cursor on the display; and store the ratio values inthe memory; wherein the face detection function is configured to: detecta face in each frame of the sequence of images; extract the detectedface; store the extracted frame in the memory; and end processing of thecurrent frame if the user's face is not detected; wherein the eyedetection function is configured to: retrieve the extracted face;localize and extract an eye region encompassing both of the user's eyesfrom the extracted face; split the eye region into a right eye regionand a left eye region; determine for each eye region an eye center and apupil center; store the eye center and pupil center for each eye regionin the memory; and detect a left eye wink, a right eye wink, or a blink,wherein the left eye wink is a closing of the left eye while the righteye remains open, the right eye wink is a closing of the right eye whilethe left eye remains open, and the blink is a closing of both the lefteye and right eye; wherein the gaze mapping function is configured to:map movement of the user's eyes to movement of the mouse cursor on thedisplay using the eye center and pupil center for each eye regionretrieved from the memory and the ratio values retrieved from thememory, wherein movement of the mouse cursor is mapped based on thelocation of one eye and the mapping is corrected using the correspondinglocalization of the second eye; perform a left mouse click when the lefteye wink has been detected for a number of frames greater than or equalto a wink threshold; perform a right mouse click when the right eye winkhas been detected for a number of frames greater than or equal to thewink threshold; and perform a double left mouse click when the blink hasbeen detected for a number of frames greater than or equal to a blinkthreshold.